Discounted Continuous-time Markov Decision Processes with Unbounded Rates: the Dynamic Programming Approach

摘要

This paper deals with unconstrained discounted continuous-time Markovdecision processes in Borel state and action spaces. Under some conditionsimposed on the primitives, allowing unbounded transition rates and unbounded(from both above and below) cost rates, we show the regularity of thecontrolled process, which ensures the underlying models to be well defined.Then we develop the dynamic programming approach by showing that the Bellmanequation is satisfied (by the optimal value). Finally, under somecompactness-continuity conditions, we obtain the existence of a deterministicstationary optimal policy out of the class of randomized history-dependentpolicies.